In a typical data processing system, data to be processed is read from an inexpensive but low speed data source and placed into a relatively expensive but high speed cache. When the data is again required, the processor first looks to the cache and, if the data is present, the processor retrieves the data rapidly. If the data is no longer in the cache, the processor must fetch it from the data source. To avoid retrieval delays, therefore, it is preferable to maintain as much data as possible in the cache. However, the capacity of the cache is limited and data is discarded periodically or when the cache approaches full capacity. A commonly used policy discards the least recently used (LRU) data first on the assumption that the more recently used data is more likely to be needed first. Another commonly used policy discards the oldest data first (first in, first out or FIFO) on the assumption that such data is least likely to be needed.
However, most conventional cache discard policies, such as the LRU policy, do not take into account that not all data takes the same amount of time to retrieve. One policy which addresses this issue is proposed in commonly-assigned U.S. Pat. No. 6,408,362 in which recently (and possibly frequently) used data, which has a very short access latency (fetch time), is discarded in favor of retaining less recently used data, which has a much longer access latency.
In addition to fetch latencies, data from different sources may have different processing priorities. Thus, when a processing node is being used to process both local and remote data, the application processing the local data (executing wire transfers for a bank) may be more important than the application processing the remote data (updating the bank's employee records). However, both an LRU discard policy and a latency-based policy may favor retaining the less important remote data in cache at the expense of the higher priority local data.